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Abstract 


The  process  of  teleoperation  can  be  described  as  allowing  a  remote  user  to  control  a  vehicle 
by  interpretting  sensor  information  captured  by  the  vehicle.  One  method  that  is  frequently  used 
to  implement  teleoperation  is  to  provide  the  user  with  a  real-time  video  display  of  a  perspective 
camera  mounted  on  the  vehicle.  This  method  limits  the  remote  user  to  seeing  the  environment 
in  which  is  vehicle  is  present  through  the  fixed  viewpoint  with  which  the  camera  is  mounted. 
Having  a  fixed  viewpoint  is  extremely  limiting  and  significantly  impedes  the  ability  of  the  remote 
user  to  properly  navigate.  One  way  to  address  this  problem  is  to  mount  the  perspective  camera 
on  a  pan-tilt  device.  This  is  rarely  done  because  it  is  expensive  and  introduces  a  significant 
increase  in  implementation  complexity  from  both  the  mechanical  and  electrical  point  of  view. 
With  the  advent  of  omnidirectional  camera  technology,  there  is  now  a  second  more  attractive 
alternative.  This  paper  describes  the  PARAROVER  ,  a  remote  controlled  vehicle  constructed  in 
the  summer  of  1998  to  demonstrate  the  use  of  omnidirectional  camera  technology  and  a  virtual 
reality  display  for  vehicular  teleoperation,  audio-video  surveillance  and  forward  reconnaissance. 
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1  Motivation 


The  concept  of  using  unmanned  vehicles  for  surveillance  and  forward  reconnaissance  by  proxy  in 
potentially  hazardous  environments  has  been  with  us  for  many  years. [3].  Recently,  the  deployment 
of  systems  to  accomplish  this  task  has  become  a  theme  of  interest  in  law  enforcement,  military  and 
space  exploration  programs. 

Unlike  traditional  reconnaissance  vehicles,  the  PARAROVER  is  designed  to  immerse  the  user  in 
the  target  area.  This  is  accomplished  using  two  key  pieces  of  technology,  an  omnidirectional  video 
camera  [2]  and  the  associated  Remote  Reality  [1]  display  system. 


Figure  1:  (a)  The  PARAROVER.  (b)  A  user  operating  the  PARAROVER.  He  is  wearing  the  Remote 
Reality  HMD  on  his  head  and  holding  the  remote  control  unit  in  his  hands.  The  unwarped  per¬ 
spective  image  that  he  sees  is  displayed  on  the  computer  monitor  near  the  middle  of  the  picture. 

While  wearing  the  Remote  Reality  HMD  (head  mounted  display),  the  operator  is  completely  im¬ 
mersed  in  the  environment  near  the  location  of  the  PARAROVER  .  The  user  has  complete  freedom 
of  viewing  direction,  as  the  omnidirectional  video  camera  mounted  on  the  PARAROVER  captures  a 
180°  by  360°  hemispherical  field  of  view.  The  PARAROVER  is  also  equipped  with  an  on-board  mi¬ 
crophone  that  permits  the  user  to  hear  any  sounds  within  the  immediate  vicinity  of  the  PARAROVER 
.  The  converse  functionality  is  provided  by  a  small  lapel  microphone  attached  to  the  front  of  the 
Remote  Reality  HMD  and  speakers  on-board  the  PARAROVER  . 

In  addition  to  applying  omnidirectional  camera  technology,  the  PARAROVER  is  designed  to  be 
compact,  low  cost  and  capable  of  quickly  maneuvering  through  hostile  environments.  The  vehicle 
itself  is  not  in  any  way  autonomous,  as  it  relies  completely  on  a  human  to  drive  it.  The  speed  at 
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which  the  PARAROVER  operates  is  comparable  to  that  of  a  human  jogging  or  walking  briskly. 


2  System  Overview 

2.1  Chassis 

The  chassis  of  the  PARAROVER  is  a  product  made  by  HPI  Racing  known  as  the  RS4  MT  Monster  Truck , 
part  number  501.  This  vehicle  chassis  was  chosen  because  it  is  made  of  ABS  plastic,  features  four- 
wheel  drive  and  has  a  very  wide  chassis.  The  fact  that  it  is  made  out  of  ABS  plastic  is  important 
because  ABS  plastic  is  easy  to  perform  the  modifications  that  were  necessary  to  mount  the  custom 
platform  on  which  the  omnidirectional  camera  is  attached. 


Remote  Control 


Velocity  Regulator  7.2V  NiCO  DC  motor 
(Speed  Control)  Battery  Pack 


Figure  2:  The  PARAROVER  chassis. 

Four  wheel  drive  was  chosen  in  order  to  give  the  PARAROVER  a  robust  drive-train  capable  of 
traversing  different  kinds  of  terrain.  The  wide  chassis  is  critical  to  system  stability  because  of  the 
use  of  the  omnidirectional  camera.  The  camera  tower  is  approximately  1.25  feet  tall,  and  hence, 
the  center  of  gravity  of  the  PARAROVER  is  significantly  altered  by  it's  presence.  Without  the  wide 
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profile,  the  vehicle  would  encounter  difficulties  performing  turns  at  anything  but  extremely  low 
velocities. 

2.2  Suspension 

The  primary  modification  to  the  after-market  vehicle  chassis  was  in  the  suspension  system.  The 
stock  chassis  was  equipped  with  oil-filled  shocks  that  were  designed  to  support  the  weight  of 
the  vehicle  itself  and  nothing  more.  The  additional  payload  of  the  custom  metal  body,  omnicam, 
wireless  video  transmitter,  FM  audio  receiver  and  speakers,  and  additional  DC  power  subsystem 
added  approximately  15  pounds  of  weight  to  the  vehicle.  This  caused  the  stock  vehicle  chassis  to 
sag  to  the  point  that  it  almost  touched  the  ground.  The  use  of  stiffer  springs  and  more  viscous 
shock  oil  as  well  as  spring  spacers  brought  the  vehicle  back  to  the  proper  height. 


(a)  (b)  (c)  (d) 


Figure  3:  (a)  The  original  springs  that  came  with  the  HPI  chassis,  (b)  The  spring  used  to  stiffen  the 
suspension,  (c)  Detail  of  front  shock  absorbers  and  (d)  rear  shock  absorbers. 


2.3  Body 

The  body  for  the  vehicle  was  custom  built  by  the  mechanical  engineering  shop  at  Columbia  Uni¬ 
versity's  Fu  Foundation  School  of  Engineering  and  Applied  Science.  Sheet  metal  was  chosen  as  the 
material  type  because  of  it's  malleability  and  tensile  strength.  The  body  is  fastened  to  the  chassis 
via  three  brass  bolts  nuts  that  rise  up  through  holes  drilled  into  the  ABS  plastic  chassis.  The  bolts 
are  designed  to  be  adjustable  and  allow  the  body  to  be  mounted  at  a  slight,  user  defined  angle. 
This  was  done  because  preliminary  tests  demonstrated  that  placing  the  omnicam  at  a  slight  angle 
forward  made  the  process  of  teleoperation  easier. 
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(a)  (b)  (c) 


Figure  4:  (a)  The  custom  platform  on  which  the  omnidirectional  camera  is  attached  to.  (b)  &  (c) 
The  bolts  on  the  chassis  which  hold  the  custom  platform  in  place. 

The  custom  body  plays  two  roles  in  the  PARAROVER  system  .  It  provides  a  mounting  bracket 
for  the  speakers  and  pocket-radio  for  the  full-duplex  audio  system  in  place  and  also  serves  as  a 
platform  on  which  the  omnidirectional  camera  is  attached.  Since  the  omnidirectional  camera  that 
was  selected  for  this  implementation  of  the  PARAROVER  had  facilities  to  be  mounted  on  a  ceiling  for 
use  as  a  security  camera,  it  was  very  easy  to  use  those  same  screw  holes  to  bolt  the  camera  onto  the 
custom  body.  Three  field  strip  screws  were  used  to  ease  disassembly  and  maintenance  procedures. 

2.4  Radio  Control  System 

The  radio  control  system  provides  the  user  with  the  interface  to  the  drive  (throttle)  and  steering 
electro-mechanics  on  the  PARAROVER  .  A  commercial  system  known  as  the  Airtronics  CX2P  was 
chosen  because  of  its  low  cost  and  use  of  FM  signal  encoding  and  transmission  in  the  75  MHz 
band.  The  FM  signal  encoding  provides  a  higher  SNR  compared  to  the  AM  encoding  that  is  more 
common  in  the  radio  controlled  vehicle  market.  In  addition,  the  use  of  the  75MHz  band  as  op¬ 
posed  to  the  25  MHz  band  was  chosen  for  its  of  higher  signal  penetration  and  better  skin  depth 
characteristics. 


2.5  Omnidirectional  Camera 

A  single  omnidirectional  video  camera  is  the  primary  sensor  that  enables  the  PARAROVER  to  be 
maneuvered  and  through  which  the  user  can  be  immersed  into  world  around  the  omnicam.  A 
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Figure  5:  The  hand-held  radio  controller  that  allows  the  user  to  cause  the  PARAROVER  to  acceler¬ 
ated,  decelerate  and  turn. 

commercial  version  of  the  omnidirectional  video  camera  [2]  called  the  Paracamera,  manufactured 
and  sold  by  Cyclovision  Technologies ,  Inc.,  was  used  in  the  implementation  of  the  PARAROVER  .  The 
omnicam  captures  a  complete  hemispherical  360  x  180  degree  picture  at  video  rate  (30  Hz).  This 
key  piece  of  technology  separates  the  PARAROVER  from  the  traditional  reconnaissance  /  surveil¬ 
lance  vehicle,  as  it  permits  the  user  to  be  immersed  in  the  environment  of  the  PARAROVER  without 
any  moving  parts. 


2.6  Wireless  Video  System 

The  wireless  video  system  allows  the  NTSC  video  signal  produced  by  the  omnicam  onboard  the 
PARAROVER  to  be  transmitted  back  to  a  remote  location  where  the  user  teleoperates  the  rover.  Since 
this  is  the  primary  method  by  which  the  user  is  able  to  maneuver  the  rover  and  ultimately  perform 
the  action  of  reconnaissance,  this  part  of  the  system  must  be  particularly  robust.  Furthermore,  the 
rover  may  not  be  within  a  line-of-sight  of  the  user.  In  fact,  there  may  be  any  number  of  obstacles 
of  various  materials  between  the  user  and  the  PARAROVER  unit,  even  if  the  actual  distance  between 
the  PARAROVER  and  user  is  small,  such  as  inside  a  building. 

A  commercial  system  manufactured  and  sold  by  "HDS,  Inc."  was  chosen  for  use  in  the  PARAROVER 
because  of  previous  experience.  The  transmitter  (marketed  by  HDS  under  the  name  "Series  0  Wire- 
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Figure  6:  The  omnidirectional  camera  that  is  mounted  on  top  of  the  PARAROVER  chassis. 

less  Video  Miniature  Transmitter")  is  extremely  small  (1.60"  x  1.60"  x  0.74")  and  draws  only  a  few 
hundred  milliamperes  of  current.  The  transmitter  is  housed  in  a  custom  housing  at  the  top  of  the 
omnicam  tower.  This  location  was  chosen  in  order  to  prevent  the  2.4  GHz  omnidirectional  antenna 
from  blocking  the  field  of  view  of  the  user.  The  corresponding  receiver  unit  is  an  HDS  "Series  7 
Wireless  Video  Receiver."  The  system  uses  FM  encoding  of  the  NTSC  video  signal  created  by  the 
omnicam.  The  wireless  link  can  be  maintained  over  a  distance  of  2  miles  in  open  space. 


2.7  Full-Duplex  Audio  System 

The  PARAROVER  is  equipped  with  a  full  duplex  audio  system  to  make  the  notion  of  immersion 
more  realistic.  While  wearing  the  Remote  Reality  HMD,  the  user  can  hear  sounds  as  if  they  were 
actually  present  at  the  physical  location  of  the  PARAROVER  .  In  addition,  any  sounds  that  originate 
from  the  vicinity  of  the  user  are  transmitted  and  played  back  on  the  speakers  attached  to  the  side 
of  the  PARAROVER  . 

A  microphone  that  is  attached  to  the  top  of  the  camera  tower  facilitates  the  capture  of  audio 
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(a)  (b) 


Figure  7:  (a)  HDS  Series  7  Wireless  Video  Receiver,  (b)  HDS  Series  0  Wireless  Video  Miniature 
Transmitter  mounted  on  top  of  the  omnidirectional  camera. 

from  the  PARAROVER  to  the  user.  Any  sound  within  the  immediate  vicinity  of  the  PARAROVER  is 
transmitted  as  a  5  MHz  FM  sub-carrier  on  the  2.4  GHz  video  signal.  This  functionality  was  built 
into  the  after-market  HDS  wireless  video  system  that  transmits  the  omnicamera  image  back  to  the 
user. 

The  reverse  process  of  transmitting  sound  from  the  user  to  the  PARAROVER  is  facilitated  by  a 
small  lapel  microphone  that  is  attached  to  the  HMD  of  the  Remote  Reality  system.  This  micro¬ 
phone  is  connected  to  a  small  pre-amplifier  that  is  attached  to  the  left  side  of  headband  that  holds 
the  HMD  in  place.  The  amplified  signal  is  then  passed  to  an  FM  transmitter  that  sends  the  signal 
using  a  commercial  FM  band  to  small  pocket  radio  mounted  on  the  underside  of  the  PARAROVER 
body.  The  pocket  radio  is  connected  to  two  speakers  that  have  been  mounted  on  either  side  of  the 
PARAROVER  body. 

2.8  DC  Power  System 

The  PARAROVER  is  powered  by  two  sets  of  batteries.  The  first  set  is  a  bundle  of  six  1.2  volt  1200 
mAH  C-size  nickel  cadnium  batteries  that  powers  the  drive-train,  steering  servo  and  remote  con¬ 
trol  electronics.  This  particular  configuration  was  chosen  because  it  is  commonly  used  in  remote 
control  vehicles,  and  hence,  battery  packs,  chargers  and  conditioners  are  easily  available.  The  7.2 
volt  battery  pack  is  held  in  place  by  a  cradle  built  into  the  HPI  chassis  that  is  specifically  designed 
for  this  purpose.  In  order  to  recharge  the  batteries,  the  operator  must  remove  the  pack  from  the 
vehicle  and  attach  it  to  a  commonly  available  7.2  volt  NiCD  charger. 


10 


(a)  (b) 


Figure  8:  (a)  The  microphone  that  is  mounted  on  top  of  the  omnidirectional  camera,  (b)  A  view  of 
the  PARAROVER  body  upside-down.  The  small  pocket  radio  can  be  seen  in  the  center  of  the  body. 


Figure  9:  The  battery  holders  embedded  into  the  base  of  the  omnidirectional  camera.  The  omnicam 
was  held  "upside-down"  while  taking  this  picture. 

The  second  set  of  batteries  powers  the  omnidirectional  camera  and  wireless  video  transmitter. 
A  separate  power  system  was  chosen  primarily  because  it  isolated  the  sensitive  eletronics  in  the 
wireless  unit  from  the  electrical  noise  generated  by  the  DC  motor.  In  order  to  satisfy  the  voltage 
and  current  requirements  of  both  the  video  camera  and  the  wireless  transmitter,  ten  1.2  volt  800 
mAH  nickel-metal-hydride  batteries  were  embedded  into  the  base  of  the  omnidirectional  camera. 
Due  to  physical  size  constraints,  battery  sockets  were  used  instead  rather  than  assembling  all  ten 
of  cells  into  a  battery  "pack"  such  as  the  one  used  to  power  the  rest  of  the  PARAROVER  .  In  order 
to  recharge  the  cells,  they  need  to  be  removed  and  placed  into  an  aftermarket  charger/ conditioner 
for  NiMH  batteries. 
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2.9  Remote  Reality  Display  System 

The  omnidirectional  video  camera  captures  an  image  from  a  parabolic  mirror  which  is  difficult  to 
interpret  unless  processed.  For  many  applications,  a  processed  image  displayed  on  the  screen  of  a 
computer  is  sufficient.  For  the  PARAROVER  ,  a  virtual  reality  system  is  used  instead. 


Figure  10:  (a)  A  image  captured  from  an  omnidirectional  video  camera  in  its  raw  form,  (b)  A 
perspective  image  generated  from  the  omnidirectional  image  data  in  (a).  This  is  similar  to  what 
the  operator  of  the  PARAROVER  would  see. 


The  Remote  Reality  [1]  system  was  developed  by  Terrance  Boult  of  Lehigh  University.  It  provides 
a  perspective  unwarping  of  the  circular  image  transmitted  by  the  PARAROVER  omnicam  with  a 
viewing  direction  that  is  based  on  the  user's  head  orientation.  In  addition,  the  HMD  has  built-in 
earphones  that  replicate  any  sound  that  is  picked  up  by  the  microphone  on  the  PARAROVER  . 


Figure  11:  The  Remote  Reality  HMD.  The  small  lapel  microphone  attached  to  it  for  the  audio  sub¬ 
system  can  be  seen  in  sitting  in  the  middle  of  the  front  visor  panel. 

The  use  of  the  Remote  Reality  makes  the  teleoperation  of  PARAROVER  a  very  natural  process.  It 
creates  the  illusion  that  the  user  is  actually  sitting  in  the  driver  seat  of  the  PARAROVER  .  The  user 
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can  look  around  at  the  surroundings  around  the  PARAROVER  by  simply  moving  his  or  her  head. 


3  Performance 

3.1  Mechanical 

Cruising  Speed  12  MPH  (7.5  f) 

Top  Speed  20  MPH  (12.5  f) 

Acceleration  4  HJHL  (2.5  p 

Turning  Radius  5  feet  (1.5  m)  minimum 
Turning  Radius  10  feet  (3  m)  at  cruise  speed 

3.2  Electrical 

Battery  Running  Time 
Battery  Running  Time 
Video  Signal  Transmission  Distance 
Video  Signal  Transmission  Distance 
Control  Signal  Transmission  Distance 
Control  Signal  Transmission  Distance 

4  Evaluation 

4.1  Successes 

Overall,  the  PARAROVER  was  well-received  at  the  various  DARPA  meetings  that  it  was  demon¬ 
strated  at.  In  addition,  the  initial  design  goals  of  demonstrating  omnicam  technology  for  teleoper¬ 
ation  and  environment  immersion,  minimizing  the  system  cost  and  attaining  better  than  average 
mechanical  performance,  were  all  met. 

The  PARAROVER  successfully  demonstrated  the  use  of  omnidirectional  video  camera  technology 
for  vehicular  teleoperation  and  target  site  immersion.  It  is  this  feature  that  sets  the  PARAROVER 
apart  from  traditional  unmanned  forward  reconnaissance  /  surveillance  devices. 

The  cost  of  the  current  PARAROVER  implementation  and  Remote  Reality  system  is  relatively  low 
(approx.  $6000)  compared  to  other  systems  of  this  type.  The  majority  of  the  cost  went  into  the 
HDS  wireless  video  system  (approximately  $3500).  This  could  have  been  alleviated  by  using  a  less 
costly  wireless  video  system  in  the  implementation,  but  the  HDS  system  was  chosen  because  of 
good  experiences  with  HDS  in  the  past  in  other  projects. 


1  hour  continuous  use 
3  hours  typical  use 
2  miles  (3.2  km)  free  space 
350  feet  (1200  m)  indoors 
1  mile  free  space 
250  feet  indoors 
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Finally,  the  PARAROVER  is  remarkable  in  that  it  is  capable  of  attaining  relatively  high  speeds 
compared  to  similar  more  expensive  devices  being  developed  at  this  time.  However,  due  to  an 
offset  center  of  gravity,  the  PARAROVER  is  limited  in  speed  while  turning.  This  was  discussed  in 
Section  4.2. 

4.2  Limitations 

The  operation  of  the  PARAROVER  is  restricted  by  a  number  of  factors  of  which  the  most  notable  are 
the  wireless  communication  systems.  Since  the  PARAROVER  is  teleoperated  by  a  remote  user,  the 
signals  from  the  remote  control  system  and  the  wireless  video  system  must  be  properly  received  by 
the  appropriate  party.  Since  the  wireless  systems  operate  on  different  electromagnetic  frequency 
bands,  the  material  penetration,  skin  depth  and  propagation  properties  of  the  signals  are  very 
different.  This  often  leads  to  a  situation  where  one  system  operates  correctly  when  another  does 
not,  resulting  in  the  user  "driving  blind"  or  being  able  to  see  but  not  control  the  PARAROVER  .  This 
is  particularly  problematic  indoors,  where  multipath  propagation  can  cause  serious  problems  for 
almost  any  kind  of  wireless  device. 

A  second  limitation  in  PARAROVER  operation  is  due  to  the  omnicam  tower.  The  Cyclovision 
Paracamera  is  more  than  1  foot  high,  with  almost  all  of  it's  weight  in  the  upper  1  /3  of  the  tower.  This 
causes  a  significant  offset  of  the  PARAROVER 's  center-of-gravity  in  the  upward  direction,  thereby 
compromising  mechanical  stability.  An  attempt  to  counter  the  offset  was  made  by  placing  the 
DC  power  cells  for  the  camera  and  wireless  video  transmitter  at  the  base  of  the  camera  tower.  In 
addition,  the  remote  controlled  vehicle  chassis  was  specifically  selected  because  of  it's  wide  wheel 
base.  Although  both  these  factors  helped  to  mechanically  stabilize  the  PARAROVER  ,  the  vehicle 
must  still  be  decelerated  to  less  than  half  of  it's  normal  operating  speed  before  making  sharp  turns. 

Finally,  the  actual  process  of  teleoperating  the  PARAROVER  is  somewhat  deceiving  in  that  it 
would  seem  to  be  straightforward  because  it  resembles  driving  an  automobile.  However,  after 
working  with  the  PARAROVER  for  some  time,  it  becomes  apparent  that  significant  training  is  re¬ 
quired  before  a  user  can  safely  and  effectively  operate  the  PARAROVER  .  The  main  reason  for  this 
is  because  the  PARAROVER  operator  sees  the  world  through  the  perspective  of  a  fictitious  person 
that  would  be  sitting  on  the  "roof"  of  the  vehicle  and  not  inside.  Driving  from  this  perspective 
is  difficult,  but  more  importantly,  it  is  not  the  same  as  driving  an  automobile.  In  particular,  col¬ 
lision  avoidance  and  traversing  terrains  with  multiple  obstacles  becomes  much  harder  and  takes 
practice. 

The  latter  two  issues  are  being  addressed  as  a  new,  enhanced  version  of  the  PARAROVER  will  be 


14 


assembled  in  March  1999.  The  second  generation  PARAROVER  will  use  a  recently  developed  folded 
omnicam.  This  new  omnicam  is  significantly  smaller  and  lighter  than  the  model  mounted  on  the 
current  PARAROVER  .  In  addition,  most  of  the  weight  in  the  new  folded  omnicam  is  at  the  base  of 
the  camera,  thus  alleviating  the  center  of  gravity  problem  with  the  current  PARAROVER  .  The  lower 
profile  of  the  new  omnicam  should  also  permit  mounting  in  such  a  way  that  the  viewpoint  is  lower. 
This  should  allow  the  user  to  see  the  PARAROVER 's  tires  and  hopefully  be  able  to  avoid  obstacles 
and  collisions  better. 


Relevant  WWW  URLs 

Columbia  Automoted  Vision  Environment 

Columbia  University 

Cyclovision  Technologies 

Defense  Advanced  Research  Projects  Agency 

DARPA  Video  Surveillance  and  Monitoring 

HDS 

HPI 
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